genai-util: refactor streams to a generic ABC wrapper.#4500
Merged
aabmass merged 40 commits intoMay 12, 2026
Merged
Conversation
There was a problem hiding this comment.
Pull request overview
This PR introduces shared synchronous/asynchronous stream wrapper base classes in opentelemetry-util-genai and refactors the OpenAI v2 chat streaming implementation to use the shared lifecycle shape (including moving chat choice buffering into a dedicated module and switching chat spans to the InferenceInvocation lifecycle).
Changes:
- Added
SyncStreamWrapper/AsyncStreamWrapperABCs inopentelemetry-util-genaiplus unit tests for stream lifecycle behavior. - Refactored OpenAI v2 chat streaming to use new
ChatStreamWrapper/AsyncChatStreamWrapperand extracted buffering intochat_buffers.py. - Added/updated OpenAI v2 tests for API exception and user exception propagation; updated changelogs.
Reviewed changes
Copilot reviewed 11 out of 11 changed files in this pull request and generated 8 comments.
Show a summary per file
| File | Description |
|---|---|
| util/opentelemetry-util-genai/src/opentelemetry/util/genai/_stream.py | Adds new generic sync/async stream wrapper base classes and lifecycle finalization logic. |
| util/opentelemetry-util-genai/tests/test_stream.py | Adds unit coverage for wrapper iteration, close/context-manager behavior, and error handling. |
| util/opentelemetry-util-genai/CHANGELOG.md | Documents the new shared stream wrapper helpers. |
| instrumentation-genai/opentelemetry-instrumentation-openai-v2/src/opentelemetry/instrumentation/openai_v2/utils.py | Switches chat invocation creation to TelemetryHandler.start_inference() / InferenceInvocation. |
| instrumentation-genai/opentelemetry-instrumentation-openai-v2/src/opentelemetry/instrumentation/openai_v2/patch.py | Updates chat create wrappers to use the new invocation lifecycle and new chat stream wrappers. |
| instrumentation-genai/opentelemetry-instrumentation-openai-v2/src/opentelemetry/instrumentation/openai_v2/chat_wrappers.py | New chat streaming wrappers built on the shared stream wrapper base classes. |
| instrumentation-genai/opentelemetry-instrumentation-openai-v2/src/opentelemetry/instrumentation/openai_v2/chat_buffers.py | Extracts choice/tool-call buffering helpers to avoid import cycles. |
| instrumentation-genai/opentelemetry-instrumentation-openai-v2/tests/test_choice_buffer.py | Updates tests to import buffers from the new module. |
| instrumentation-genai/opentelemetry-instrumentation-openai-v2/tests/test_chat_completions.py | Adds tests for API exception propagation and streaming user-exception propagation. |
| instrumentation-genai/opentelemetry-instrumentation-openai-v2/tests/test_async_chat_completions.py | Adds async variants of exception propagation tests; adjusts stream close behavior by mode. |
| instrumentation-genai/opentelemetry-instrumentation-openai-v2/CHANGELOG.md | Documents the chat stream wrapper refactor. |
lmolkova
reviewed
Apr 30, 2026
xrmx
changed the title
aabmass
reviewed
May 6, 2026
…b.com/eternalcuriouslearner/opentelemetry-python-contrib into feat/move-stream-to-genai-util-as-abc
…b.com/eternalcuriouslearner/opentelemetry-python-contrib into feat/move-stream-to-genai-util-as-abc
JamieDanielson
approved these changes
May 12, 2026
github-project-automation
Bot
moved this from Ready for merge
to Done
in Python PR digest
6 tasks
7 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
This change introduces shared stream wrapper base classes in util-genai and updates the OpenAI v2 chat streaming wrappers to use that common lifecycle shape.
The new SyncStreamWrapper and AsyncStreamWrapper centralize stream iteration, close/context-manager handling, passthrough behavior, double-finalization protection, and failure handling. OpenAI chat streaming now uses dedicated ChatStreamWrapper and AsyncChatStreamWrapper classes that mirror the Messages and Responses wrapper structure while continuing to propagate SDK/API exceptions and user exceptions unchanged.
This also moves chat choice buffering into a separate chat_buffers.py module to avoid import cycles, switches the new chat path to LLMInvocation/InferenceInvocation lifecycle methods, and adds tests for stream wrapper behavior and sync/async chat exception propagation.
Fixes # (issue)
Type of change
Please delete options that are not relevant.
How Has This Been Tested?
Does This PR Require a Core Repo Change?
Checklist:
See contributing.md for styleguide, changelog guidelines, and more.